Detecting Off-topic Responses to Visual Prompts

Automated methods for essay scoring have made great progress in recent years, achieving accuracies very close to human annotators. However, a known weakness of such automated scorers is not taking into account the semantic relevance of the submitted text. While there is existing work on detecting answer relevance given a textual prompt, very little previous research has been done to incorporate visual writing prompts. We propose a neural architecture and several extensions for detecting off-topic responses to visual prompts and evaluate it on a dataset of texts written by language learners

Online representation learning in recurrent neural language models

© 2015 Association for Computational Linguistics. We investigate an extension of continuous online learning in recurrent neural network language models. The model keeps a separate vector representation of the current unit of text being processed and adaptively adjusts it after each prediction. The initial experiments give promising results, indicating that the method is able to increase language modelling accuracy, while also decreasing the parameters needed to store the model along with the computation required at each step

Unsupervised Entailment Detection between Dependency Graph Fragments

Entailment detection systems are generally designed to work either on single words, relations or full sentences. We propose a new task – detecting entailment between dependency graph fragments of any type – which relaxes these restrictions and leads to much wider entailment discovery. An unsupervised framework is described that uses intrinsic similarity, multi-level extrinsic similarity and the detection of negation and hedged language to assign a confidence score to entailment relations between two fragments. The final system achieves 84.1% average precision on a data set of entailment examples from the biomedical domain

Auxiliary Objectives for Neural Error Detection Models

We investigate the utility of different auxiliary objectives and training strategies within a neural sequence labeling approach to error detection in learner writing. Auxiliary costs provide the model with additional linguistic information, allowing it to learn general-purpose compositional features that can then be exploited for other objectives. Our experiments show that a joint learning approach trained with parallel labels on in-domain data improves performance over the previous best error detection system. While the resulting model has the same number of parameters, the additional objectives allow it to be optimised more efficiently and achieve better performance

Compositional sequence labeling models for error detection in learner writing

© 2016 Association for Computational Linguistics. In this paper, we present the first experiments using neural network models for the task of error detection in learner writing. We perform a systematic comparison of alternative compositional architectures and propose a framework for error detection based on bidirectional LSTMs. Experiments on the CoNLL-14 shared task dataset show the model is able to outperform other participants on detecting errors in learner writing. Finally, the model is integrated with a publicly deployed self-assessment system, leading to performance comparable to human annotators

Parser lexicalisation through self-learning

We describe a new self-learning framework for parser lexicalisation that requires only a plain-text corpus of in-domain text. The method first creates augmented versions of dependency graphs by applying a series of modifications designed to directly capture higherorder lexical path dependencies. Scores are assigned to each edge in the graph using statistics from an automatically parsed background corpus. As bilexical dependencies are sparse, a novel directed distributional word similarity measure is used to smooth edge score estimates. Edge scores are then combined into graph scores and used for reranking the topn analyses found by the unlexicalised parser. The approach achieves significant improvements on WSJ and biomedical text over the unlexicalised baseline parser, which is originally trained on a subset of the Brown corpus

Health-Related Quality of Life in Portuguese Patients with Chronic Hepatitis C

INTRODUCTION: Chronic hepatitis C virus (HCV) infection impacts multiple health and psychosocial dimensions and encompasses a significant overall burden as it progresses to advanced stages of hepatic disease. AIMS: To evaluate for the first time health-related quality of life (HRQoL) of a subset of Portuguese adult patients with chronic hepatitis C using the Portuguese versions of generic, Short-Form 12 Health Survey (SF-12v2), and disease-specific, Chronic Liver Disease Questionnaire (CLDQ), instruments; to assess psychometric properties of CLDQ, Portuguese version. METHODS: HRQoL was evaluated in Portuguese adult outpatients with chronic hepatitis C attending the Hepatology Clinic at Centro Hospitalar do Porto, using SF-12v2 and CLDQ. This transversal study was conducted between April and October 2015. RESULTS: Eighty outpatients with chronic hepatitis C were enrolled, with mean age 57 years (standard deviation 11), 67.5% male, all Caucasian, 76.3% diagnosed for >10 years, 66.3% with C virus genotype 1, 65.0% with hepatic cirrhosis (94.2% of which Child-Pugh A), and 46.3% under current antiviral treatment. For CLDQ internal consistency, Cronbach's α was 0.88; for construct validity, correlations ranged from 0.36 to 0.80 (p < 0.01). Mean CLDQ scores ranged from 4.25 (Worry) to 5.78 (Abdominal Symptoms). Lower scores were observed for Worry, Fatigue, and Emotional Function domains. Statistically significant differences were found in median values of Worry (CLDQ) and Role Emotional (SF-12) (p < 0.05) for "current antiviral treatment," with higher scores for patients that concluded therapy. CONCLUSION: HRQoL was negatively affected in several domains in Portuguese patients with chronic hepatitis C; oral antiviral treatment correlated with better quality of life, assuring its benefits on this population; the CLDQ Portuguese version revealed adequate psychometric properties, and was useful in assessing quality of life in Portuguese HCV patients.info:eu-repo/semantics/publishedVersio

Attending to characters in neural sequence labeling models

Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words. We investigate character-level extensions to such models and propose a novel architecture for combining alternative word representations. By using an attention mechanism, the model is able to dynamically decide how much information to use from a word- or character-level component. We evaluated different architectures on a range of sequence labeling datasets, and character-level extensions were found to improve performance on every benchmark. In addition, the proposed attention-based architecture delivered the best results even with a smaller number of trainable parameters

Scoring lexical entailment with a supervised directional similarity network

Scoring Lexical Entailment with a Supervised Directional Similarity NetworkERC Nvidi

Automatic text scoring using neural networks

Automated Text Scoring (ATS) provides a cost-effective and consistent alternative to human marking. However, in order to achieve good performance, the predictive features of the system need to be manually engineered by human experts. We introduce a model that forms word representations by learning the extent to which specific words contribute to the text’s score. Using Long-Short Term Memory networks to represent the meaning of texts, we demonstrate that a fully automated framework is able to achieve excellent results over similar approaches. In an attempt to make our results more interpretable, and inspired by recent advances in visualizing neural networks, we introduce a novel method for identifying the regions of the text that the model has found more discriminative.This is the accepted manuscript. It is currently embargoed pending publication